Add multiline logging interceptor sample for Datadog compatibility #249
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Implements surgical solution for formatting multiline exception logs as single-line JSON - Only affects logging within Temporal SDK boundaries, doesn't interfere with global formatters - Captures exceptions in both activities and workflows using interceptor pattern - Formats exceptions as JSON with message, type, and traceback fields - Includes comprehensive sample with activities, workflows, worker, and starter - Follows established interceptor patterns from sentry sample - Solves Datadog multiline log parsing issues for Temporal exceptions Requested by: @deepika-awasthi Link to Devin run: https://app.devin.ai/sessions/37122594b322437b8e42cef51d4104d1 Co-Authored-By: deepika awasthi <deepika.awasthi@temporal.io>
…ckages - Fixed import sorting in worker.py, starter.py, workflows.py, test_imports.py - Added multiline_logging to pyproject.toml packages list for proper build inclusion - All black formatting already applied in previous commit Co-Authored-By: deepika awasthi <deepika.awasthi@temporal.io>
- Changed from absolute import 'from activities import' to relative import 'from .activities import' - This resolves mypy errors: Module 'activities' has no attribute 'complex_failing_activity' and 'failing_activity' - Follows the same import pattern used in workflows.py and other sample files - All local lint checks now pass: black, isort, and mypy Co-Authored-By: deepika awasthi <deepika.awasthi@temporal.io>
…for interceptor and workflows - Changed 'from interceptor import' to 'from .interceptor import' - Changed 'from workflows import' to 'from .workflows import' - This resolves pytest collection error: ModuleNotFoundError: No module named 'interceptor' - All imports in test_imports.py now use relative imports consistently - Local lint and test checks pass completely Co-Authored-By: deepika awasthi <deepika.awasthi@temporal.io>
Author
|
not needed |
deepika-awasthi
deleted the
devin/1727124409-multiline-logging-interceptor
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add multiline logging interceptor sample for Datadog compatibility
Summary
Adds a new sample demonstrating how to format multiline exception logs as single-line JSON within Temporal SDK boundaries. This solves the issue where multiline tracebacks span multiple log entries in log aggregation systems like Datadog, making them difficult to parse and analyze.
The solution implements a surgical interceptor that:
message,type, andtracebackfieldsThe implementation follows the established interceptor pattern from the sentry sample.
Review & Testing Checklist for Human
temporal server start-dev) and verify multiline exceptions are logged as single-line JSON in worker outputworkflow.unsafe.sandbox_unrestricted()usage in the workflow interceptor is appropriate and follows Temporal best practicesNotes
Testing limitations: The implementation wasn't fully tested in a live Temporal environment due to temporal CLI not being available in the development environment. The core JSON formatting logic was validated, but full interceptor integration testing is needed.
Performance impact: The interceptor adds JSON serialization overhead on every exception, but this should be minimal since exceptions are rare in normal operation.
Requested by: @deepika-awasthi
Link to Devin run: https://app.devin.ai/sessions/37122594b322437b8e42cef51d4104d1